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I.  INTRODUCTION 


Trust  bonds  relationships,  both  personal  as  well  as  professional.  Within  an 
organization,  trust  facilitates  sharing  of  information,  improved  morale,  flexibility,  and 
open  communication.  It  can  enhance  efficiency  by  reducing  the  need  for  governance,  a 
time  consuming  effort  for  managers  and  supervisors  (Van  de  Ven,  2004). 

Fukuyama  (1995)  states  the  United  States  was  one  of  the  first  countries  to  develop 
large,  modern  professionally  managed  corporations.  This  innovative  creation  of  larger 
organizations  was  possible  because  the  American  culture  allowed  organizations  to  build 
networks  beyond  family  ties,  relying  instead  on  the  socially  bred  high  degree  of  trust 
among  fellow  citizens. 

The  trust  that  helped  establish  larger  organizations  has  been  declining  over  the 
past  five  decades  according  to  Bruhn  (2001).  He  reviewed  numerous  polls  and  studies  on 
trust,  honesty,  integrity,  and  commitment,  and  concluded,  “Today,  nearly  two  in  three 
Americans  believe  that  most  people  can’t  be  trusted;  half  say  most  people  would  cheat 
each  other  if  they  had  a  chance,  and  half  say  that  most  people  are  looking  out  for 
themselves  (Bruhn,  2001).” 

Just  as  trust  in  U.S.  society  is  changing,  there  are  also  new  developments  in 
organizations.  A  diversity  of  organizational  structures  continues  to  emerge  and  change 
and  managers  are  no  longer  limited  to  the  traditional  hierarchical  organization. 
Technological  advances,  globalization,  competition,  and  workforce  dynamics  have  driven 
these  emerging  and  changing  organizational  structures. 

Organizational  performance  is  impacted  by  the  combination  of  these  two 
changing  factors,  trust  and  structure.  This  project  analyzes  the  effects  of  varying  trust 
conditions  and  organizational  structure  on  perfonnance. 

The  remainder  of  this  section  discusses  differences  in  organizational  structure  and 
trust,  and  concludes  with  a  broad  description  of  this  project’s  research  goal. 
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A.  ORGANIZATIONAL  STRUCTURES 

Two  primary  structures  are  formal  Hierarchy  and  flat,  organic  structures,  also 
known  as  Edge  structures  (Alberts  &  Hayes,  2003).  Bolman  and  Deal  (2003)  suggest 
that  organizations  are  arranged  to  minimize  problems  while  maximizing  performance. 
They  offer  assumptions  underlying  structural  design.  First,  organizations  exist  to  achieve 
goals  and  objectives.  Next,  they  seek  efficiency  and  enhanced  performance  through 
specialization  and  division  of  labor.  Lastly,  the  optimal  form  of  coordination  ensures  that 
efforts  of  the  organization’s  members  and  units  mesh  properly. 

Mintzberg  (1979)  describes  the  basic  blueprint  of  an  organization  through  by 
depicting  five  different  components:  the  strategic  apex,  the  technostructure,  the  support 
staff,  the  middle  line,  and  the  operating  core.  Organizational  structures  differ  from  one 
another  through  different  distributions  and  emphasis  of  each  of  the  components.  An 
introduction  to  the  hierarchy  is  described  in  Mintzberg ’s  models  of  the  simple  structure 
and  the  machine  bureaucracy. 

The  simple  structure  consists  of  only  two  components,  the  strategic  apex  and  the 
operating  core.  The  leadership  at  the  strategic  apex  provides  decisions  and  infonnation, 
which  flow  downward,  to  the  workforce  in  the  operating  core.  The  top  tier  provides 
monitoring  and  direct  supervision.  Members  within  the  core  communicate  upward  with 
the  leadership  rather  than  with  one  another.  This  arrangement  is  often  used  in  an 
environment  involving  simple  and  routine  tasks,  and  a  single  person  directs  the  work  of 
the  organization  (Mintzberg,  1979).  Startup  companies  often  begin  with  a  simple 
structure;  an  example  is  the  business  operated  in  a  garage  started  by  William  Hewlett  and 
David  Packard  (Bolman  &  Deal,  2003). 

The  machine  bureaucracy  is  comprised  of  many  layers  between  the  strategic  apex 
and  the  operating  core.  The  added  layers  of  the  machine  bureaucracy  are  found  in  the 
middle  line  where  management  operationalizes  the  decisions  from  the  apex,  distributes 
information,  and  provides  supervision  to  members  of  the  operating  core.  The  structure 
also  incorporates  large  support  staffs  and  technostructure.  Like  the  simple  structure,  the 
machine  bureaucracy  is  typically  used  in  organizations  where  tasks  are  routine.  Unlike 
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the  simple  structure,  the  machine  bureaucracy’s  top  tier  is  concerned  with  larger  strategic 
objectives  than  supervision  (Mintzberg,  1979).  McDonald’s  organization  incorporates  an 
example  of  this  structure  (Bolrnan  &  Deal,  2003). 

The  fonnal  Hierarchy  seeks  to  achieve  control  and  stability,  characterized  with 
clear  lines  of  authority  and  a  “chain  of  command”  (Ancona,  Kochan,  Scully,  Van  Maanen 
&  Westney,  1999).  This  traditional  structure  has  contended  with  new  organizational 
forms  over  the  past  20  years  because  of  changing  and  dissolving  organizational 
boundaries  as  a  result  of  technological  advances  (Wigand,  2008),  pressures  of 
globalization,  competition,  and  workforce  dynamics  (Bolrnan  &  Deal,  2003).  The  new 
forms  emphasize  a  flattening  of  organizational  structure,  enhancement  of  networked 
capabilities,  achievement  of  changing  objectives  through  flexibility  of  operations,  and 
reliance  on  diverse  skills  (Ancona  et  ah,  1999). 

A  new  structure  that  has  emerged  is  the  Edge.  This  organizational  design 
leverages  networking  by  moving  knowledge  and  power  among  geographically  distributed 
participants  to  the  edges  of  organizations  (Leweling  &  Nissen,  2007).  The  Edge  seems 
appropriate  for  the  military  because  of  its  broadening  involvement  in  a  variety  of  military 
operations  beyond  combat.  Examples  include  humanitarian  assistance,  homeland 
defense,  and  noncombatant  evacuations  (U.S.  Department  of  Defense,  2006).  These 
expanding  mission  sets  require  organizations  to  operate  beyond  routine  tasks  and 
transition  to  capabilities  requiring  flexibility. 

B.  COALITIONS 

In  the  context  of  increasing  technologies  and  global  interaction,  organizations 
increase  in  complexity,  and  therefore,  collaborate  with  other  organizations  to  accomplish 
goals  requiring  numerous  functions  and  specialties.  Task  forces  and  teams  assemble  to 
harness  the  various  capabilities  of  different  organizations  (Bolrnan  &  Deal,  2003).  The 
military  uses  the  concept  of  assembling  organizations  as  evidenced  by  its  involvement  in 
numerous  multinational  coalitions  (U.S.  Department  of  Defense,  2000),  interagencies 
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(U.S.  Department  of  Defense,  2006),  and  interorganizational  networks  (Kennedy,  2004; 
Erckenbrack  &  Scholer,  2004).  An  example  of  U.S.  Department  of  Defense  participation 
in  a  multinational  coalition  is  the  Multi  National  Force-Iraq,  MNF-I  (MNF-I,  2008). 

The  MNF-I  mission  is  to  work  in  partnership  with  the  Iraqi  Government  to 
contribute  to  the  maintenance  of  security  and  stability  of  the  country.  Twenty-four 
countries  participate  in  the  coalition.  The  Multi  National  Core-Iraq,  MNC-I,  an 
organization  within  the  coalition,  is  the  tactical  unit  responsible  for  command  and  control 
operations  in  Iraq.  The  organization  is  geographically  distributed  in  six  regions 
throughout  the  country.  Each  region  maintains  a  multinational  force  and  an  associated 
headquarters  (MNF-I,  2008). 

Both  the  Edge  and  Hierarchy  organizations  are  incorporated  in  the  MNF-I 
coalition  (MNF-I,  2008).  The  two  forms  are  illustrated  through  differing  forms  of 
Australian  participation  in  the  coalition.  An  example  of  the  Edge  organization  is  the 
Coalition  Counter  Improvised  Explosive  Device  Task  Force.  A  small  number  of 
Australian  Defense  Force  personnel  work  alongside  members  from  other  nations’ 
military  forces  to  coordinate  intelligence  collection  and  Iraqi  Forces  training  in  support  of 
the  counter  IED  efforts  (Australian  Government,  Department  of  Defense,  2008).  The 
Australians  also  maintain  a  Hierarchy  within  the  coalition.  The  HMAS  Parramata  is  an 
Australian  frigate,  which  patrols  the  Northern  Persian  Gulf  to  assist  in  the  protection  of 
Iraq’s  offshore  assets  such  as  oil  platforms  (Australian  Government,  Department  of 
Defense,  2008).  The  Australian  crew  aboard  the  Parramata  is  organized  in  the  traditional 
hierarchy  organization  whereby  the  Commanding  Officer  maintains  authority  at  the 
strategic  apex  and  makes  the  decisions  about  the  ship’s  operations.  He  distributes 
information  downward  through  the  department  heads  in  the  middle  line  of  the 
organizational  structure. 

C.  TRUST  WITHIN  COALITIONS 

Trust  is  a  critical  element  in  achieving  a  coalition’s  goals  and  objectives.  Trust  is 
required  because  of  interdependence  and  diversity  (Mayer,  Davis,  &  Schoorman,  2005). 
The  increasing  complexities  of  organizations  results  in  interdependence  on  other 
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organizations,  and  therefore,  vulnerabilities  arise  due  to  reliance  on  the  other  participants. 
Organizations  within  coalitions  must  therefore  trust  that  other  organizations  will  perforin 
actions  important  to  them  (Mayer  et  ah,  1995).  Coalitions  are  composed  of  numerous 
participants  and  units  resulting  in  increased  diversity.  Members  of  the  coalition  cannot 
rely  on  interpersonal  or  interorganizational  similarities  and  common  background  and 
experience  to  enhance  the  willingness  to  work  together  (Berscheid  &  Walster,  1978; 
Newcomb,  1956),  and  therefore,  must  develop  trust  as  a  mechanism  to  enable  them  to 
work  together  effectively.  Efficiency,  perfonnance,  and  coordination  hinge  on  trust  to 
ensure  the  elements  of  interdependence  and  diversity  enhance  a  coalition’s  operations, 
rather  than  hinder  it. 

D.  PURPOSE  OF  TRUST  RESEARCH  WITHIN  THE  CONTEXT  OF 

DIFFERENT  ORGANIZATIONS 

Military  organizations  have  adapted  over  many  centuries  (Winnefeld  &  Johnson, 
1993;  Allard,  1996;  Palmer,  2005;  Leighton,  1952).  Military  hierarchy  organizations 
have  enjoyed  refinement  through  combat,  training,  and  doctrinal  development  (Janowitz, 
1959).  In  contrast,  applications  of  the  Edge  structure  within  military  organizations  have 
not  been  tested  due  to  the  recent  introduction  of  the  structure  (Nissen,  2005).  Increasing 
levels  of  interorganizational  coordination  within  the  military  indicate  that  Edge 
organizations  may  be  more  suitable  than  Hierarchy  command  and  control  structures  in 
certain  circumstances  (Leweling  &  Nissen,  2007;  Powley,  Fry,  Barrett,  &  Bright,  2004). 

The  opportunity  exists  to  add  to  a  body  of  knowledge  concerning  the  comparative 
advantages  for  Edge  organizations  using  laboratory  experimentation,  structured  controls, 
and  data  collection  and  analysis.  This  research  pertains  to  the  efficacy  of  the  Edge 
organization  within  the  context  of  trust  and  mistrust  conditions,  set  against  the 
effectiveness  of  a  hierarchy  organization  to  provide  a  comparative  analysis. 
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Literature  focused  on  organizational  structures  provided  the  groundwork  upon 
which  the  research  hypotheses  were  based.  The  research  design  is  detailed  and  key 
findings  and  results  offered.  This  research  project  closes  with  conclusions  and 
recommendations  for  future  experimentation  to  build  upon  research  concerning 
organizational  design  and  trust  conditions. 

The  following  section  is  a  literature  review  and  discusses  existing  and  current 
theories,  methodologies,  implementing  trust  and  its  influence  on  trust  conditions  in 
organizations.  The  emphasis  is  on  trust  factors  affecting  performance  on  Hierarchical 
and  Edge  organization  types. 
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II.  LITERATURE  REVIEW 


This  section  provides  an  overview  of  existing  as  well  as  current  theories, 
methodologies,  and  implementation  of  trust  (i.e.,  integrity,  ability,  and  benevolence)  and 
how  it  influences  conditions  of  trust  in  organizations.  It  also  focuses  on  how  the  factors 
of  trust  affect  the  performance  in  Hierarchal  Organization  types  as  well  as  Edge 
Organization  types  with  the  specific  task  of  solving  complex  problems  with  a  high  rate  of 
success.  Even  though  the  factors  included  in  the  section  are  separate  and  individually 
relevant  in  establishing  trust  in  small  group  dynamics,  each  factor  works  together  to  form 
the  system  necessary  for  success  in  small  organizations.  This  section  concludes  by  the 
integration  of  trust,  and  its  effects  on  military  style  command  and  control  (C2)  situations 
as  well  as  autonomous  Edge  Organization  types.  It  will  also  include  the  positive  and 
negative  results  of  Hierarchical  Organization  and  Edge  Organization  types. 

A.  RELEVANCE  OF  TRUST  LINKED  TO  PERFORMANCE 

The  role  of  trust  in  groups  whose  goal  is  to  achieve  a  high  level  of  success  while 
working  in  groups  has  been  extensively  studied.  Most  of  the  studies  were  performed  in  a 
Hierarchical  Organization  type  environment  similar  to  those  used  in  military  settings. 
One  central  figure  had  overall  control  of  the  organization  with  a  series  of  subordinate 
entities  with  specific  tasks  within  the  system.  All  control  decisions  were  executed  by  the 
central  control  figure. 

Trust  in  the  top  is  a  significant  determinant  in  how  the  subordinate  group 
performs  its  tasks  and  how  successful  the  results  of  these  groups  ultimately  impacted. 
The  following  factors  determine  what  contributes  to  trust  or  mistrust:  information, 
influence  and  control. 

An  authority  figure  that  does  not  trust  subordinates  disseminates  information 
significantly  differently  than  an  authority  figure  that  does.  Someone  who  does  not  trust 
other  members  of  the  team/unit  will  not  be  completely  forthcoming  with  all  relevant 
information  needed  for  the  individual  to  minimize  exposure  to  the  rest  of  the  group 


7 


(Zand,  1972).  The  non-trusting  member  does  not  tend  to  want  relevant  contributions 
from  other  members  of  the  group  to  exert  their  influence  further.  Although  the  non¬ 
trusting  member  exerts  influence  in  tenns  of  the  proposal  of  goals,  suggestions  for 
reaching  goals  and  metrics  for  evaluating  success,  two-way  communications  from  other 
members  of  the  team  are  highly  discouraged.  Yet,  the  non-trusting  member  expects 
subordinates  to  accept  inputs  readily  (Zand,  1972). 

“Finally,  one  who  does  not  trust  will  try  to  minimize  his  dependence  on  others. 
He  will  feel  he  cannot  rely  on  other  members  of  the  group  to  abide  by  agreements  and 
will  try  to  impose  controls  on  their  behavior  when  coordination  is  necessary  to  attain 
common  goals,  but  will  resist  and  be  alarmed  at  their  attempts  to  control  his  behavior” 
(Zand,  1972,  p.  230).  When  low-trust  is  established  within  a  group,  all  the  necessary 
means  of  communication  such  as  dependence  on  other  members  of  the  group, 
coordination  of  information,  adequate  feedback  from  all  associated  members,  and  the 
need  to  attain  a  common  goal  ultimately  leads  to  decreased  ability  to  solve  problems 
efficiently. 

Groups  comprised  of  individuals  with  adequate  levels  of  trust  among  its  members 
act  differently  and  solve  problems  more  effectively  than  groups  with  low  levels  of  trust. 
Groups  that  display  high  levels  of  trust  are  quantitatively  better  able  to  solve  problems 
within  the  group  because  the  members  are  more  willing  to  be  vulnerable  when  it  comes 
to  problem-solving  efforts.  Barriers  to  information  dissemination,  accuracy,  relevance, 
and  timeliness  are  not  prevalent  when  groups  have  higher  levels  of  trust  since 
interdependence  and  reception  to  the  influence  of  others  is  encouraged  for  timely 
accurate  problem  solving.  “They  will  also  accept  interdependence  because  of  confidence 
that  others  will  control  their  behavior  in  accordance  with  agreements,  (see  Figure  1)  and 
therefore  will  have  less  need  to  impose  controls  on  others”  (Zand,  1972,  p.  231).  Groups 
with  high  levels  of  trust  tend  to  be  more  creative  problem  solvers  with  long-range 
relevant  solutions  compared  with  their  low-trust  counterparts. 
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Figure  1.  A  Model  of  the  Relationship  of  Trust  to  Information,  Influence  and 
Control,  (From:  Zand,  1972). 


Establishment  of  trust  is  a  significant  and  relevant  element  in  Edge  Organization 
types  as  well  as  Hierarchy  organization  types.  Ability,  benevolence,  and  integrity  are 
equally  significant  tangible  aspects  that  help  build  trust  and  promote  its  use  throughout 
the  life  of  the  organization  type  of  choice,  be  it  Edge  or  Hierarchy.  Ability,  benevolence, 
and  integrity  are  individually  beneficial  when  attempting  to  establish  trust  in  a  relatively 
short  period  of  time;  however,  when  ability,  benevolence,  and  integrity  are  combined 
holistically,  each  personality  trait  adds  value  and  validity  to  the  individual  or  individuals 
comprising  the  organization  when  establishing  trust  is  paramount. 


B.  ABILITY 


The  domain  of  the  ability  is  specific  because  the  trustee  may  be  highly  competent 
in  some  technical  area,  affording  that  person  trust  on  tasks  related  to  a  specific  area. 
However,  the  trustee  may  have  little  aptitude,  training,  or  experience  in  another  area,  for 
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instance,  in  interpersonal  communication.  Although  such  an  individual  may  be  trusted  to 
perform  analytic  tasks  related  to  the  technical  area,  the  individual  may  not  be  trusted  to 
initiate  contact  with  an  important  customer  (Mayer,  Davis,  &  Schoonnan,  1995). 

The  use  of  ability  in  the  study  of  Hierarchical  Organization  types  can  be  difficult 
to  quantify  since  ability  is  a  quality  that  tends  to  be  very  specific  and  not  always  able  to 
be  used  outside  of  the  member’s  specialty  or  area  of  expertise.  As  discussed  earlier, 
ability,  benevolence,  and  integrity  are  units  of  measure  or  defining  factors  of  group 
behavior  that  as  a  system  lead  to  rapid  development  of  trust  in  small  groups  to  effectively 
solve  problems.  Ability  is  a  significant  measure  in  the  system  of  developing  trust; 
however,  ability  tends  to  be  a  very  specific  aspect  in  establishing  competence.  Even 
though  an  individual  may  have  a  high  level  of  ability  in  one  area  such  as  quantum 
physics,  the  same  individual  may  not  have  any  aptitude,  training,  or  experience  in  a  field 
such  as  the  social  dynamics  that  determine  the  selection  of  targets  by  terrorist 
organizations.  For  instance,  a  terrorist  of  Hispanic  descent  may  pick  a  different  target  to 
destroy  than  a  terrorist  of  Arab  descent  based  on  the  culture  in  which  the  terrorist  was 
raised.  The  social  dynamics  of  Latin  American  cultures  may  dictate  that  a  military  target 
is  more  significant  to  western  societies  and  thus  choose  it  as  a  viable  target  that  focuses 
on  western  centers  of  gravity.  On  the  other  hand,  a  terrorist  of  Arabic  descent  may 
determine  that  targets  of  economic  importance  may  be  more  significant  to  western 
societies.  Although  such  an  individual  may  be  trusted  to  do  one  job  in  an  area  of 
expertise,  trust  to  perform  tasks  in  areas  outside  the  area  of  expertise  may  not  be  granted. 
Thus,  trust  is  domain  specific  (Zand,  1972). 

C.  BENEVOLENCE 

Benevolence  is  a  trait  that  is  very  specific  to  a  situation  or  job  being  perfonned, 
and  thus,  changes  with  each  individual  situation.  Benevolence  in  a  military  setting  may 
be  viewed  completely  differently  than  benevolence  in  a  civilian  business  setting. 
Benevolence  is  also  specific  to  individuals.  Another  individual  in  the  same  organization 
type  may  see  what  one  person  deems  to  be  benevolent  behavior  as  not  benevolent  but 
instead  the  behavior  could  be  viewed  as  neutral  or  even  malevolent.  Benevolence 
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focuses  on  the  relationship  of  the  trustor  and  the  trustee,  how  the  relationship  affects 
performance,  and  explains  the  potential  long-tenn  positive  effects  that  benevolence  can 
have  in  a  relationship  where  gaining  and  maintaining  trust  is  paramount.  Benevolence 
suggests  that  the  trustee  has  some  specific  attachment  to  the  trustor  (Mayer,  Davis,  & 
Schoonnan,  1995). 

An  example  of  this  relationship  can  be  found  in  many  interactions  in  the  military 
as  well  as  civilian  organizations.  Mentoring  or  apprenticeships  are  relationships  that 
require  a  sense  of  benevolence  on  the  behalf  of  the  trustee.  There  is  no  benefit  for  the 
trustee  being  helpful  to  the  trustor;  however,  if  the  trustee  is  benevolent,  the  desire  is  to 
see  the  trustor  succeed  with  no  reward  or  benefit  on  the  part  of  the  trustee.  “Benevolence 
is  the  perception  of  a  positive  orientation  of  the  trustee  toward  the  trustor”  (Mayer,  Davis, 
&  Schoorman,  1995,  p.  718).  Benevolence  in  work  group  relationships  can  be  a 
cornerstone  of  trust  in  a  working  relationship  of  this  type  because  the  trustor  perceives 
that  the  trustee  is  helping  out  of  no  benefit  to  himself,  but  of  a  genuine  want  or  need  to 
help  the  new  individual  become  a  contributing  member  of  the  group  or  team. 

D.  INTEGRITY 

Integrity  is  difficult  to  measure  because  perceptions  can  change  from  place  to 
place  or  organization  to  organization.  For  instance,  subordinates,  peers  and  superiors 
perceive  military  officers  in  the  environment  of  a  military  organization  to  have  inherently 
high  levels  of  integrity.  This  integrity  is  based  on  naval  traditions  of  behavior,  personal 
characteristics,  and  honesty  of  all  naval  officers.  However,  in  a  different  organization, 
the  standards  that  dictate  a  military  officer’s  integrity  may  be  seen  as  a  negative  in  a 
different  organization.  Therefore,  the  perceived  integrity  of  the  same  individual  could  be 
greatly  diminished  dependent  on  the  organizational  circumstances.  A  naval  officer  with 
high  levels  of  integrity  may  not  be  seen  in  the  same  vein  when  applying  for  a 
management  position  at  Enron. 

In  conclusion,  there  are  several  factors  to  consider  when  attempting  to  establish 
trust  with  the  mission  of  correctly  and  efficiently  solving  complex  problems.  Without 
properly  establishing  trust  (i.e.,  ability,  benevolence,  and  integrity)  the  ability  of  the 
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group  to  perform  tasks  could  be  greatly  diminished,  leading  to  the  unsuccessful 
completion  of  the  objective.  Although  Hierarchy  Organization  and  Edge  Organization 
types  are  dissimilar  in  how  they  are  managed,  it  is  believed  that  high  levels  of  trust, 
(based  on  these  dimensions)  will  lead  to  successful  problem  solving  in  both  organization 
types. 

E.  HYPOTHESES 

In  this  project,  considering  trust  and  its  effect  on  edge  organization  types  as  well 
as  effects  on  hierarchy  organization  types,  we  broke  out  the  two  organizations  based  on 
two  measurable  trust  conditions.  Our  first  condition  was  the  organization  type;  which  are 
edge  and  hierarchy,  and  whether  the  trust  condition  affects  solution  outcomes.  Second, 
we  measured  the  trust  conditions  and  their  affects  on  solution  outcomes.  The  first  set  of 
hypotheses  suggests  a  flat,  organic  organizational  form  will  perform  better  than  a 
hierarchical  form. 

The  hypotheses  were  as  follows: 

•  Hypothesis  1:  The  Edge  organization  type  will  outperform  Hierarchy 
organization  type  regardless  of  trust  condition.  (Main  Effect) 

•  Hypothesis  1(a):  The  Edge  trust  condition  will  outperform  Hierarchy 
trust  condition.  (Interaction) 

•  Hypothesis  1(b):  The  Edge  mistrust  condition  will  outperform  Hierarchy 
mistrust  condition.  (Interaction) 

In  the  second  set  of  hypotheses,  we  suggest  that  performance  in  the  trust  (over  the 
mistrust)  condition  will  be  higher. 

•  Hypothesis  2:  Trust  will  outperform  mistrust  regardless  of  organization 
type.  (Main  Effect) 

•  Hypothesis  2(a):  Trust  in  the  Edge  organization  type  will  outperform 
mistrust  in  Edge  organization  type.  (Interaction) 

•  Hypothesis  2(b):  Trust  in  the  Hierarchy  organization  type  will  outperform 
mistrust  in  Hierarchy  organization  type.  (Interaction) 
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The  following  section  discusses  the  methodology  used  in  this  project.  It  also 
mentions  the  subjects  and  the  groups  used.  The  different  communication  protocols 
employed  are  elaboration  upon  as  well. 
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III.  METHODOLOGY 


This  section  discusses  the  methodology  utilized  in  this  project.  In  this  section,  we 
describe  the  research  design  and  the  structure  of  the  experiments.  Building  directly  upon 
research  by  Leweling  and  Nissen,  and  drawing  on  the  literature  of  Zand,  and  Mayer 
Davis  and  Schoonnan,  we  use  the  ELICIT  multiplayer  intelligence  game  to  test  subjects’ 
ability  to  solve  problems  under  combined  treatments  of  trust  and  organizational  structure. 
This  section  begins  with  a  description  of  the  ELICIT  environment  followed  by  a  detailed 
review  of  the  subjects,  groups,  and  protocols  used  in  the  experiments. 

A.  ELICIT  ENVIRONMENT 

This  experimental  design  uses  the  ELICIT  multiplayer  intelligence  game  to  place 
the  subjects  in  a  collaborative  problem  solving  environment.  Subjects  log  in  and  play  the 
game  via  client  applications  on  separate,  networked  computer  workstations.  ELICIT 
provides  subjects  with  the  challenge  of  identifying  all  of  the  key  elements  of  a  terrorist 
plot  in  an  attempt  to  prevent  the  attack  from  occurring.  Its  design  is  similar  to  the  Parker 
Brothers’  board  game  “Clue”  in  that  it  requires  each  player  to  analyze  clues  and  combine 
assessments  with  other  players  to  answer  the  following  questions. 

Who:  Who  planned/led  the  effort  to  attack?  Who  will  execute  the  attack? 

Where:  In  what  country  will  the  attack  take  place? 

What:  What  is  the  specific  target  or  event  to  be  attacked? 

When:  The  three  factors  of  “when”  are  month,  day  and  time  of  the  attack. 

Each  of  the  68  clues  called  “factoids”  describe  a  finite  detail  of  the  terrorist  plot, 
but  each  factoid  is  incomplete  in  addressing  only  one  aspect  (who,  what,  where,  when), 
and  not  every  factoid  is  necessary  or  helpful  in  assembling  an  accurate  and  complete 
answer.  The  factoids  are  distributed  to  the  17  players  as  follows:  two  factoids  are  released 
to  each  player  as  game  play  begins  and  another  after  an  additional  five  minutes  of  game 
play.  At  twenty  minutes,  all  factoids  have  been  distributed,  and  all  necessary  facts 
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regarding  the  terrorist  plot  are  available  for  solution.  No  single  player  can  solve  the 
problem  with  the  given  factoids  set  so  the  players  must  collaborate  to  solve  the  plot  and 
win  the  game. 

The  subjects  interact  with  ELICIT  via  a  website  based  display  which  shows 
various  user  status  frames.  The  list  screen  displays  all  factoids  that  a  particular  player  has 
received.  After  the  initial  factoid  distribution,  a  player’s  List  screen  would  display  the 
two  factoids  distributed  by  the  server,  for  example,  “The  Lion  is  involved,”  and,  “The 
Violet  Group  only  operates  in  Daylight.”  These  functions  are  associated  with  four, 
separate  windowed  screens,  each  corresponding  to  the  pertinent  questions  (i.e.,  who, 
what,  where,  when)  regarding  the  terrorist  plot.  One  tabbed  screen  exists  for  viewing  and 
accumulating  each  category  of  factoid  and  includes  information  regarding  who,  what, 
where  and  when  data.  During  game  play,  each  subject  has  access  to  a  set  of  five 
functional  capabilities  supported  by  the  network  client  that  allows  the  user  to  take  one  of 
five  actions  with  the  accumulated  factoids:  1)  List,  2)  Post,  3)  Pull,  4)  Share,  and  5) 
Identify. 

•  List  creates  a  list  of  chosen  factoids  selected  by  the  player  viewable  in  the 
main  game  window. 

•  Post  enables  a  player  to  have  one  or  more  factoids  displayed  on  a  common 
screen  that  can  be  viewed  by  all  other  players  simultaneously.  This 
represents  one  of  two  mechanisms  for  sharing  information  in  the  game 
(e.g.,  verbal  communication  is  prohibited  in  this  experiment  protocol). 

•  Pull  represents  the  complement  to  Post.  Any  player  can  add  posted 
factoids  to  the  List  pane. 

•  Share  represents  the  second  mechanism  for  sharing  information  in  the 
game,  and  enables  players  to  send  factoids  only  and  directly  to  one 
another. 

•  Identify  represents  the  manner  in  which  subjects  publish  their  “solutions” 
to  the  problem,  once  they  have  adequate  factoids  to  propose  a  complete 
answer  (i.e.,  who,  what,  where,  when)  regarding  the  terrorist  plot. 

Multiple  scenarios  of  game  play  are  possible  in  ELICIT.  Each  session  is 
structurally  similar  but  contains  a  different  answer  set.  Each  scenario  set  includes  17 
players  (and  pseudonyms)  and  a  set  of  68  factoids.  However,  the  factoids  and  the 
sequence  of  distribution  are  unique  to  each  version.  The  potential  exists  to  play  the  game 
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multiple  times,  even  with  the  same  group  of  subjects.  After  the  game  is  complete,  the 
moderator  shuts  down  the  server  application,  and  exportable  log  files  are  created  for 
extraction  and  analysis.  The  log  files  consist  of  time-stamped  entries  for  every  data 
transaction  in  the  game  including  game  start,  players’  names  (pseudonyms),  details  of 
factoid  distribution  (what  specific  factoid  and  to  whom),  all  interactions  between  players 
including  Post,  Pull,  Share,  Identify,  and  game  end. 

B.  SUBJECTS 

Subjects  for  this  study  are  135  first-quarter  graduate  students  at  the  Naval 
Postgraduate  School  (NPS)  enrolled  in  a  core  organizational  behavior  course.  Students  in 
the  Graduate  School  of  Business  and  Public  Policy  (GSBPP)  consist  of  active  duty 
members  of  the  United  States  Army,  Air  Force  Navy  and  Marine  Corps  as  well  as  foreign 
students  from  the  School  of  International  Graduate  Studies  (SIGS).  A  total  of  135 
subjects  possess  between  one  and  18  years  of  work  experience.  All  subjects  have 
undergraduate  college  degrees  as  well  as  direct  military  service.  This  representative 
sample  serves  to  enhance  the  external  validity  of  the  study. 

Despite  the  considerable  level  of  realism  designed  into  the  ELICIT  game,  the 
information- sharing  and  processing  task  is  limited  in  the  scope  of  communication  and 
interpersonal  interaction.  It  is  also  structured  so  that  subjects  can  be  evaluated  in  as  little 
as  45  minutes  of  game  play.  The  networked-computer,  ELICIT-mediated  task 
environment  does  not  enable  all  of  the  typical  communication  modalities  such  as 
telephone,  video  teleconference,  face-to-face  communication,  email  and  group 
interaction.  This  structure  serves  to  enhance  internal  validity  through  control  but  limits 
the  external  validity  of  the  study  as  applied  to  more  typical  group  communication. 

C.  GROUPS 

Subjects  were  randomly  assigned  to  one  of  eight  groups  of  17  members  each.  A 
list  of  subjects  was  ranked  in  decreasing  order  of  seniority  to  ensure  an  equal  distribution 
of  age  and  experience  among  the  eight  groups.  Each  group  contained  an  equally 
distributed  representation  of  military  service  branch,  organizations,  officer  subspecialties, 
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genders,  and  country  of  service  to  accurately  create  the  conditions  present  in  Joint  and 
Coalition  military  organizations.  Subjects  were  assigned  to  one  of  the  eight  work  groups 
as  follows. 


Test  day  one 

A:  Hierarchy,  Trust 

B:  Edge,  Trust 

C:  Hierarchy,  Mistrust 

D:  Edge,  Mistrust 


Test  day  two 

A:  Hierarchy,  Trust 

B:  Edge,  Trust 

C:  Hierarchy,  Mistrust 

D:  Edge,  Mistrust 


Figure  2.  Treatment  Matrix 

The  most  senior  subjects  were  assigned  to  the  role  of  “Alex,”  the  leader  in  the 
ELICIT  game.  This  is  similar  to  the  manner  in  which  professional  analysts  are  assigned 
to  specific  roles  in  operational  intelligence  organizations  in  the  field,  and  hence  helps  to 
ground  this  experiment  though  confonnance  to  practice.  This  approach  contrasts  a  little 
with  that  of  randomized  assignment  imposed  in  some  prior  studies,  emphasizing  a 
concern  for  realism  over  replication. 

D.  PROTOCOLS 


Subjects  reported  to  a  network  classroom  on  their  assigned  day  for  the 
experiment.  Subjects  were  assigned  to  play  specific  roles  (e.g.,  as  identified  via 
pseudonyms)  in  the  game.  Subjects  were  briefed  about  the  experiment  in  their 
organizational  behavioral  class  and  formally  consented  to  participate  in  it.  When  all 
students  were  seated  and  the  ELICIT  clients  connected  with  the  server,  subjects  sat  down 
at  the  appropriate  workstations,  were  informed  verbally  about  the  nature  of  the 
experiment,  and  asked  to  read  a  set  of  instructions  pertaining  to  both  the  experiment  and 
the  ELICIT  environment.  Subjects  had  10  minutes  to  read  the  instruction  set  quietly. 
Subjects  were  encouraged  to  ask  questions  about  the  functional  nature  of  the 
experimental  settings  throughout  this  process.  Once  subjects  read  the  instructions  they 
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were  offered  a  ten-minute  break  and  were  encouraged  to  discuss  their  approach  to  the 
scenario.  At  the  end  of  the  break,  the  students  took  their  places  at  the  workstations  and 
the  moderator  began  the  simulation. 

1.  Hierarchy 

The  hierarchy  group  is  stratified  into  three  functional  tiers.  The  Senior  Supervisor 
(Alex,  labeled  “1”)  may  share  factoids  with  Middle  Managers  (i.e.,  labeled  “2,”  “6,” 
“10,”  “14”)  at  any  time.  Middle  Managers  may  share  factoids  with  the  Senior  Supervisor, 
other  Middle  Managers  or  their  subordinate  team  Operators  (labeled  “3,”  “4,”  “5”  in  the 
“2”  group  and  so  forth).  Operators  may  share  factoids  with  other  operators  in  their  team 
and  with  the  Middle  Managers.  No  one  may  post  globally  (which  would  share  factoids 
with  ALL  individuals),  and  each  solution  group  in  the  hierarchy  may  only  Pull  factoids 
pertaining  to  their  specific  group’s  task,  who,  what,  where,  and  when.  This  manipulation 
administers  the  nature  of  the  Hierarchy  represented  in  Figure  2  and  follows  that  of 
Leweling  &  Nissen,  2007. 

“A  Senior  Supervisor  (i.e.)  is  responsible  for  the  intelligence  organization  as  a 
whole,  and  has  four  Middle  Managers  reporting  directly.  Each  such  leader  in  turn  has 
three  Operators  reporting  directly,  and  is  responsible  for  one  set  of  details  associated  with 
the  terrorist  plot.  For  instance.  Middle  Manager  2  and  team  would  be  responsible  for  the 
“who”  details  (e.g.,  which  terrorist  organization  is  involved)  of  the  plot,  Middle  Manager 
6  and  team  would  be  responsible  for  the  “what”  details  (e.g.,  what  the  likely  target  is), 
and  so  forth  for  “when”  and  “where.”  Subjects  are  shown  this  organization  chart,  told  of 
their  responsibilities  within  the  organization,  and  provided  with  a  short  description  of  the 
hierarchy.  Additionally,  the  ELICIT  software  limits  subjects’  Post  and  Pull  access  to 
specific  common  screens  within  this  manipulation.  Specifically,  those  players  in  the 
“who”  group,  for  instance,  are  allowed  to  Post  to  and  Pull  from  only  one  of  the  four 
common  screens  (i.e.,  the  “who”  screen)  noted  above.  Comparable  restrictions  apply  to 
players  in  the  other  three  functional  groups.  The  only  exception  applies  to  the  Senior 
Supervisor  1,  who  has  post-pull  access  to  all  four  common  screens.” 
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Who  Group  What  Group  Where  Group  When  Group 

Figure  3.  Hierarchy 

2.  Edge  Organization 

The  Edge  Organization  is  very  different.  At  any  time,  any  subject  assigned  to  an 
Edge  organization  may:  a)  share  factoids  with  any  other  member;  b)  post  factoids  to,  c) 
pull  factoids  from  any  solution  set,  Who,  What,  Where  and  When;  or  d)  Identify  with 
partial  or  complete  answers  to  who,  what,  where,  or  when  in  the  terrorist  plot.  In  the  Edge 
organization  manipulation,  there  are  no  pre-assigned  leaders  or  functional  groups 
established  in  advance  of  the  experiment.  As  noted  above,  the  players  are  pre-assigned  to 
specific  roles  (i.e.,  pseudonyms)  within  the  game,  but  the  various  roles  reflect  no 
hierarchical  or  functional  differences  from  one  another.  Subjects  are  told  about  the 
functional  design  in  an  Edge  organization  and  informed  that  they  may  self-organize  as 
they  see  best  to  solve  the  test  case.  Figure  2  characterizes  the  nature  of  this  Edge 
manipulation. 
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Figure  4.  Edge  Organization 

Without  an  overall  leader  or  functional  groups,  subjects  must  decide  for 
themselves  who  works  on  which  aspects  of  the  problem,  and  who  posts,  pulls  and 
exchanges  information  with  whom.  Edge  subjects  are  encouraged  to  discuss  their 
approach  to  the  problem  during  the  ten-minute  break.  With  the  Edge  manipulation,  the 
ELICIT  software  does  not  limit  subjects’  Post  and  Pull  access  to  specific  common 
screens,  and  any  player  can  post  to  and  pull  from  any  of  the  four  common  screens  (i.e., 
“who,”  “what,”  “when,”  “where”).  Consistent  with  the  other  manipulation  is  the  Share 
function,  through  which  any  player  can  share  factoids  directly  with  any  other.  The  game 
ends  after  45  minutes  of  game  play  and  all  players  identify  the  plot  details.  The 
moderator  prompts  all  subjects  to  identify  with  their  best  assessment  before  the  end  of  the 
simulation. 
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3. 


Trust 


The  trust  manipulation  used  in  the  experiment  is  independent  of  the  organization 
manipulation  summarized  above.  It  is  based  on  the  three  perceived  factors  of  trust 
expressed  in  Mayer,  Davis  and  Schoorman’s  study.  Trust  as  a  treatment  is  communicated 
to  the  subjects  through  expressions  that  communicate  high  levels  of  benevolence,  ability 
and  integrity  in  the  instructions  to  subjects.  See  Appendices  A  and  B  for  details. 

Benevolence  is  demonstrated  with  the  statement,  “Members  of  your  community 
share  information  freely  with  a  general  orientation  toward  doing  good  to  others.  We  are 
impressed  with  this  orientation  and  are  encouraged  by  the  positive  interactions  among 
your  fellow  cohort  members.”  Ability  is  applied  to  all  subjects  with  the  suggestion  that, 
“your  intellect,  varying  skills,  and  past  experience  lead  us  to  believe  that  you  are  well 
qualified  to  solve  the  terrorist  threat  problem.”  Integrity  exists  in  the  protocol  and 
treatment  by  confirming  that,  “your  actions  will  be  consistent,  congruent,  and  credible 
with  established  protocols  and  guidelines.” 

4.  Mistrust 

The  mistrust  manipulation  used  in  the  experiment  is  independent  of  the 

organization  manipulation  summarized  above  also.  It  is  based  on  undennining  the  three 

perceived  factors  of  trust  expressed  in  Mayer,  Davis  and  Schoorman’s  study,  or  showing 

an  absence  of  these  factors.  Mistrust  as  a  treatment  is  communicated  to  the  subjects 

through  expressions  that  communicate  low  levels  of  benevolence,  ability  and  integrity  in 

the  instructions  to  subjects.  See  Appendices  C  and  D  for  details.  The  protocol  item  that 

impedes  benevolence  is  the  expression,  “Members  of  your  community  normally  work 

well  together,  but  frequently  withhold  information  from  each  other.  We  are  unsure  about 

how  you  interact  among  your  fellow  cohort  members  and  question  whether  negative 

interactions  have  affected  your  relationships”  and  “previous  sessions  reveal  that  some 

individuals  take  pride  in  undennining  team  cohesion  and  effectiveness  by  generating  and 

releasing  false  information  or  by  non-participation  in  the  exercise.”  Similarly,  the 

subjects’  ability  is  put  under  suspicion  by  suggesting  that  “we  have  yet  to  assess  your 

intellect  and  skills,  and  wonder  whether  past  experience  qualifies  you  to  solve  the 
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terrorist  threat  problem  as  a  group.”  Last,  the  protocol  attacks  the  group  and  their 
individual  integrity  by  announcing  that  “we  are  discouraged  that  when  it  comes  to 
solving  critical  problems  in  group  settings  such  as  this  that  your  actions  may  not  be 
consistent,  congruent,  and  credible  with  established  protocols  and  guidelines.  Simply  put, 
be  wary  of  moles  and  free-riders.” 

Research  suggests  that  in  organizations  with  high  levels  of  perceived 
benevolence,  ability  and  integrity,  an  environment  of  trust  can  and  usually  does  develop. 
In  the  absence  of  these  factors,  mistrust  fills  the  vacuum. 

The  following  section  discusses  data  analysis  and  the  results.  It  also  mentions  the 
correlation  analysis  conducted,  the  multivariate  analysis  done,  the  perfonnance, 
interactive  effects,  and  finally,  trust  conditions. 
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IV.  ANALYSIS  AND  RESULTS 


This  section  reviews  the  results  from  the  ELICIT  simulation.  We  examine  the 
performance  data  of  individual  players  in  the  four  conditions  to  determine  1)  any 
correlations  between  performance  parameters,  2)  factors  that  may  significantly  affect 
performance  among  the  various  treatments,  and  3)  any  comparisons  of  performance 
among  the  different  treatments.  This  section  begins  with  a  summary  of  the  descriptive 
statistics.  The  next  segment  examines  the  correlation  between  the  dependent  variables, 
(accuracy  scores  and  speed  scores).  The  next  two  segments  examine  a  multivariate 
analysis  followed  by  univariate  analyses.  The  section  then  provides  a  summary  of 
support  for  the  research  hypotheses.  Finally,  the  section  concludes  with  key  findings. 
The  results  of  our  analysis  offer  insight  into  preferable  combinations  of  organization 
types  and  trust  conditions  to  optimize  effectiveness. 

A.  ANALYSIS  AND  DESCRIPTIVE  STATISTICS 

The  primary  analysis  consists  of  comparing  the  mean  scores  of  the  different 
treatments.  The  analysis  is  conducted  using  mean  scores  from  the  two  measures  of 
problem  solving  effectiveness  (the  dependent  variables):  individual  accuracy  scores  and 
individual  speed  scores.  Table  1  shows  the  descriptive  statistics  of  individual 
performance,  displaying  the  mean  results  and  standard  deviations  within  each  group.  The 
first  four  rows  of  the  table  correspond  to  the  four  treatment  groups  and  reflect  interaction 
effects  between  the  manipulations  of  organization  type  and  trust  condition.  The  next  two 
rows  correspond  to  main  effects  of  the  organization  manipulation,  holding  the  trust 
condition  constant.  The  last  two  rows  correspond  to  main  effects  of  the  trust 
manipulation,  holding  the  organization  type  constant. 


25 


Organization  Type 

Trust  Condition 

N 

Accuracy  Mean 

Accuracy  Std.  Deviation 

Speed  Mean 

Speed  Deviation 

Edge 

High 

34 

0.55 

0.05 

0  25 

0  03 

Edge 

Low 

33 

0.33 

0.04 

0.12 

0  01 

Hierarchy 

High 

30 

0.32 

0.05 

0.09 

0.02 

Hierarchy 

Low 

34 

0.34 

0.05 

0.23 

0  02 

Edge 

High  +  Low 

67 

0.44 

0.03 

0.19 

0.17 

Hierarchy 

High  +  Low 

64 

0.33 

0.03 

0.16 

0.18 

Edge  +  Hierarchy 

High 

64 

0.44 

0.04 

0.17 

0.02 

Edge  +  Hierarchy 

Low 

67 

0.34 

0.03 

0.18 

0.02 

Table  1.  Descriptive  Statistics 

B.  CORRELATION  BETWEEN  DEPENDENT  VARIABLES 

A  correlation  analysis  is  first  conducted  to  determine  the  relationship  between 
individual  accuracy  scores  and  individual  speed  scores.  A  positive  correlation  is  found 
between  the  two  scores,  as  illustrated  in  Table  2.  This  shows  that  individuals  scored 
higher  on  accuracy  if  they  answered  earlier  in  the  experiment,  while  accuracy  scores 
decreased  as  the  simulation  approached  the  45-minute  termination  point.  This  result 
seems  counter  intuitive  because  one  would  expect  that  more  time  spent  working  on  a 
problem  results  in  more  accurate  answers.  The  positive  correlation  is  a  result  of  the 
requirement  that  every  participant  had  to  provide  an  answer  prior  to  the  conclusion  of 
each  experiment.  Consequently,  approximately  78%  of  participants  answered  within  the 
last  10  minutes  of  the  experiment.  The  “late”  answering  participants  scored  a  mean 
accuracy  of  .34  compared  to  the  rest  of  the  participants  who  answered  earlier  and  scored 
a  mean  accuracy  of  .55.  This  result  implies  that  the  participants  who  answered  during  the 
last  10  minutes  of  the  experiment  probably  did  so  at  the  urging  of  the  administrators  and 
not  because  they  had  correctly  assembled  a  complete  solution.  Because  the  dependent 
variables  are  correlates  of  one  another,  a  multivariate  analysis  is  appropriate. 
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Performance  Measure 

Speed 

Accuracy 

Speed 

Pearson  Correlation 

1 

.281  (**) 

Sig  (2 -tailed) 

0001 

N 

135 

135 

Accuracy 

Pearson  Correlation 

281  (**) 

1 

Sig.  (2 -tailed) 

0.001 

N 

135 

135 

Table  2.  Correlation  Between  Performance  Measures 

Given  that  the  simulation  occurred  over  a  two-day  period,  we  test  whether  the 
mean  scores  from  day  1  differed  from  those  of  day  2.  We  use  an  analysis  of  variance 
(ANOVA)  to  examine  those  differences.  We  find  that  in  terms  of  accuracy,  there  is  no 
significant  difference.  However,  in  terms  of  speed,  we  find  that  day  2  was  faster  with 
marginal  significance  at  the  .  1  level,  again  suggesting  that  the  administrators  encouraged 
participants  to  identify  and  finish  the  simulation  slightly  more  rapidly  than  on  day  1 . 

C.  MULTIVARIATE  ANALYSIS 


Next,  we  perfonn  a  multivariate  analysis  of  variance  (MANOVA).  Table  3  shows 
the  summary  results.  The  main  effect  of  our  Organization  Type  manipulation  is 
significant  at  the  0.05  level,  and  the  main  effect  of  our  Trust  manipulation  is  significant  at 
the  0.1  level.  The  interaction  effect  of  our  combined  Organization  Type  and  Trust 
manipulation  is  highly  significant.  The  interaction  between  organization  type  and  trust 
appears  to  be  powerful. 


Effect 

Value 

F 

Hypothesis  df  Error  df 

Significance 

Organization  Type 

Pillia's  Trace 

0  045 

3  082 

2 

130 

0.049 

Wilk's  Lambda 

0.955 

3.082 

2 

130 

0  049 

Hotelling's  Trace 

0  047 

3  082 

2 

130 

0.049 

Roy's  Largest  Root 

0  047 

3.082 

2 

130 

0.049 

Trust  Condition 

Pillia's  Trace 

0  036 

2.407 

2 

130 

0.094 

Wilk's  Lambda 

0.964 

2.407 

2 

130 

0.094 

Hotelling's  Trace 

0.037 

2.407 

2 

130 

0.094 

Roy's  Largest  Root 

0  037 

2.407 

2 

130 

0  094 

Organization  Type  * 

Pillia's  Trace 

0202 

16.457 

2 

130 

0.000 

Trust  Condition 

Wilk's  Lambda 

0  798 

16457 

2 

130 

0.000 

Hotelling's  Trace 

0.253 

16.457 

2 

130 

0.000 

Roy's  Largest  Root 

0253 

16.457 

2 

130 

0.000 

Design:  Intercept+Organization  Type+Trust  Condition  +  Organization  Type  ’  TrustType 


Table  3.  Multivariate  Results 
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D.  UNIVARIATE  ANALYSES 


We  also  examine  how  speed  and  accuracy  vary  separately  across  our 
manipulations  through  a  series  of  Factorial  ANOVA  calculations.  Table  4  shows  the 
results  of  the  ANOVA  using  speed  scores  as  the  dependent  variable.  The  ANOVA 
reveals  that  taken  independently  the  main  effects,  (organization  type  and  trust  condition), 
are  not  significant;  however,  the  interaction  between  the  two  main  effects  is  highly 
significant,  (p<.001). 


Source 

Type  III 
Sum  of 
Squares 

df 

Mean 

Square 

F 

Significance 

Corrected  Model 

586(a) 

3 

0.195 

10.681 

0 

Organization  Type 

0.022 

1 

0.022 

1.22 

0.271 

Trust  Condition 

0.001 

1 

0.001 

0.051 

0.821 

Organization  Type  * 

Trust  Condition 

0.561 

1 

0.561 

30.661 

0.000 

Error 

2.397 

131 

0.018 

R  Squared  =  .197  (Adjusted  R  Squared  =  .178) 
DependentVariable:  Speed 


Table  4.  Univariate  Results  -  Speed 

Figure  5  delineates  the  results  of  the  mean  speed  scores.  As  summarized  in  the 
table  above,  for  a  given  level  of  trust  (i.e.,  trust  or  mistrust  condition),  speed  performance 
across  the  organization  manipulation  does  not  appear  to  vary  much.  The  same  is  apparent 
for  speed  across  the  trust  manipulation  for  a  given  organization  type  (i.e.,  Hierarchy  or 
Edge).  However,  notice  that  the  lines  in  the  figure  cross,  reflecting  substantial  variation 
when  both  organization  type  and  trust  condition  are  considered.  This  depicts  the  highly 
significant  interaction  effect  noted  above.  When  the  level  of  trust  is  low,  the  Hierarchy 
outperforms  the  Edge  in  terms  of  speed.  It  appears  as  though  the  hierarchical  organization 
structure  enables  participants  to  overcome  the  mistrust  and  work  comparatively  more 
quickly  than  in  the  Edge  despite  a  low-trust  environment.  Alternatively,  when  the  level  of 
trust  is  high,  the  Edge  outperforms  the  Hierarchy.  It  appears  as  though  the  Edge 
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organization  structure  enables  participants  to  work  much  more  quickly  than  in  the 
Hierarchy  when  trust  is  high.  Notice  that  the  Edge  organization  in  the  trust  condition 
produces  the  highest  overall  performance  in  terms  of  speed. 


Figure  5.  Mean  Speed  Scores 

Table  5  shows  results  using  accuracy  scores  as  the  dependent  variable.  Notice  in 
this  case  that  both  main  effects  are  significant  at  the  0.05  level,  and  the  interaction  effect 
is  significant  at  the  0.01  level.  Unlike  the  analysis  above,  in  which  neither  main  effect  is 
significant  in  terms  of  speed  as  a  performance  measure,  both  the  organization  type  and 
trust  condition  have  strong  influences  on  performance  in  terms  of  accuracy.  Like  the 
analysis  above,  the  interaction  of  organization  and  trust  is  strong. 
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Source 

Type  III 
Sum  of 
Squares 

df 

Mean 

Square 

F 

Significance 

Corrected  Model 

1.287(a) 

3 

0.429 

5.825 

0.001 

Organization  Type 

0.429 

1 

0.429 

5.818 

0.017 

Trust  Condition 

0.325 

1 

0.325 

4.406 

0.038 

Organization  Type  * 

Trust  Condition 

0.528 

1 

0.528 

7.161 

0.008 

Error 

9.652 

131 

0.074 

R  Squared  =  .197  (Adjusted  R  Squared  =  .178) 


Table  5.  Univariate  Results  -  Accuracy 

Figure  6  delineates  the  results  of  the  mean  accuracy  scores.  Their  interpretation  is 
a  bit  more  complicated  than  that  for  speed  outlined  above.  When  the  level  of  trust  is  low 
(i.e.,  the  mistrust  manipulation),  there  is  negligible  performance  differential  between 
organizational  types  in  terms  of  accuracy.  When  mistrust  pervades,  the  organization  type 
does  not  appear  to  make  much  difference.  Alternatively,  when  the  level  of  trust  is  high, 
The  Edge  organization  outperforms  the  Hierarchy.  Interestingly,  in  the  Hierarchy  it  does 
not  appear  to  matter  whether  trust  is  present  or  not  in  terms  of  accuracy;  performance  is 
roughly  the  same  across  both  trust  and  mistrust  conditions.  Notice  too  how  the 
combination  of  Edge  organization  type  and  trust  condition  produces  the  highest  overall 
performance  in  terms  of  accuracy.  This  parallels  the  result  in  terms  of  speed  noted  above. 
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Figure  6.  Mean  Accuracy  Scores 

E.  SUPPORT  FOR  HYPOTHESES 

Table  6  summarizes  results  in  terms  of  the  hypotheses.  Each  hypothesis  is  listed 
in  the  first  column,  and  the  corresponding  statistical  support  in  terms  of  multivariate  and 
univariate  analyses  is  noted  across  the  other  columns. 
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Hypothesis 

Statistical  Support 

MAN OVA 

ANOVA  -  speed 

ANOVA  -  accuracy 

Effect 

1.  Edge  Organization  Type 
outperforms  Hierarchy  Organization 
Type  regardless  of  trust  condition. 

supported 

not  supported 

supported 

(main  effect) 

1(a).  Edge  high  trust  condition 
outperforms  Hierarchy  high  trust 
condition. 

supported 

supported 

(interaction  effect) 

1(b).  Edge  mistrust  condition 
outperforms  Hierarchy  mistrust 
condition. 

not  supported 

not  supported 

(interaction  effect) 

2.  Trust  outperforms  mistrust 
regardless  of  organizational  type. 

supported 

not  supported 

supported 

(main  effect) 

2(a).  Trust  in  Edge  Organization 

Type  outperforms  mistrust  in  Edge 
Organization  Type. 

supported 

supported 

(interaction  effect) 

2(b).  Trust  in  Hierarchy  Organization 
Type  outperforms  mistrust  in 

Hierarchy  Organization  Type. 

not  supported 

not  supported 

(interaction  effect) 

Table  6.  Hypothesis  Statistical  Support  Summary 

We  find  that  Hypothesis  1  is  partially  supported.  The  multivariate  main  effect  for 
organization  type  is  statistically  significant,  but  univariate  results  are  mixed:  insignificant 
in  terms  of  speed  but  significant  in  terms  of  accuracy.  Hypothesis  1(a)  suggests  that  the 
Edge  Trust  Condition  outperforms  the  Hierarchy  Trust  Condition.  The  evidence  as 
indicated  by  Tables  4  and  5  and  Figures  2  and  3  supports  this  hypothesis.  Hypothesis  1(b) 
suggests  that  the  Edge  Mistrust  Condition  outperfonns  the  Hierarchy  Mistrust  Condition. 
The  evidence  as  indicated  by  Table  4  and  5  and  Figures  2  and  3  does  not  support  this 
hypothesis. 

We  find  that  Hypothesis  2  is  also  partially  supported.  The  multivariate  main  effect 
for  trust  is  statistically  significant,  but  univariate  results  are  mixed:  insignificant  in  terms 
of  speed  but  significant  in  terms  of  accuracy.  Hypothesis  2(a)  suggests  that  the  Edge 
High  Trust  Condition  will  outperform  the  Edge  Low  Trust  Condition.  The  evidence  as 
indicated  by  Tables  4  and  5  and  Figures  2  and  3  supports  this  hypothesis.  Hypothesis 
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2(b)  suggests  that  the  Hierarchy  High  Trust  Condition  will  outperform  the  Hierarchy  Low 
Trust  Condition.  The  evidence  as  indicated  by  Table  4  and  5  and  Figures  2  and  3  does  not 
support  this  hypothesis. 

F.  KEY  FINDINGS 

This  project’s  laboratory  experiment  and  analysis  consider  two  primary  concepts: 
optimal  organization  type  and  optimal  trust  condition. 

The  data  provide  mixed  support  for  Hypothesis  1.  We  find  that  within  a  High 
Trust  environment,  the  Edge  Organization  outperfonns  the  Hierarchy  Organization. 
However,  in  a  Low  Trust  environment,  no  difference  occurs  in  performance  between  the 
two  organizations.  The  data  also  provide  mixed  support  for  Hypothesis  2.  This  finding 
intimates  the  unexpected  result  whereby  trust  conditions  facilitate  differences  in 
performance  only  within  an  Edge  Organization.  Additionally,  the  highest  levels  of 
performance,  in  terms  of  both  speed  and  accuracy,  are  observed  in  the  Edge  High  Trust 
condition. 

These  results  can  provide  three  important  insights  for  organizational  designers 
when  considering  accuracy  as  a  measure  of  performance.  First,  the  Edge  Organization  is 
very  sensitive  to  trust,  while  the  Hierarchy  Organization  is  not.  This  results  because  a 
Hierarchy  Organization  operates  on  authority-driven  decisions,  rules,  policy  adherence, 
and  skill  sets  assigned  within  a  chain  of  command;  members  of  an  Edge  Organization,  on 
the  other  hand,  accomplish  tasks  relying  on  one  another’s  skills  and  knowledge.  These 
relationships  are  strengthened  in  environments  with  high  levels  of  trust.  Second,  in  a 
Trust  Condition,  the  Edge  Organization  is  the  preferable  structural  design  for  optimal 
performance.  Third,  in  the  Hierarchy,  one  can  expect  similar  perfonnance  in  either  the 
Trust  or  Mistrust  Condition. 

Section  V  builds  upon  the  results  of  this  analysis  to  offer  implications  for  practice 
and  recommendations  for  leaders,  managers,  and  organizational  designers.  The  section 
also  provides  recommendations  for  further  research  in  order  to  expand  the  body  of 
knowledge  pertaining  to  organizations  and  implications  of  trust. 
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V.  DISCUSSION,  RECOMMENDATIONS,  AND  CONCLUSIONS 


In  this  section,  we  discuss  implications  of  our  laboratory  experiments  studying  the 
effects  of  trust  and  organizational  type  on  problem  solving  effectiveness,  and  make 
recommendations  for  future  research. 

A.  DISCUSSION 

This  section  discusses  the  usefulness  of  the  experimental  results  for  both 
organizational  design  and  trust  conditions.  Additionally,  some  recommendations  for 
future  research  are  made. 

This  research  is  partly  an  attempt  to  illustrate  better  how  Edge  organizations  can 
improve  effectiveness  in  some  military  decision  making  environments.  Edge  is  an 
appropriate  structure  for  problem-solving  in  temporary  teams.  In  the  ELICIT  electronic 
environment,  Edge  has  the  best  communication  capability  for  effectiveness.  The  Edge 
structure  benefits  from  the  decentralization  of  power  and  influence,  minimal  formality 
and  omni-directional  information  flow.  The  results  support  the  research  hypothesis  that  in 
a  High  Trust  condition,  an  Edge  organization  accesses  faster  and  more  accurate  problem 
solving  capability  by  its  flexible  and  self-organizing,  responsive  structure.  However, 
Edge  organizations  appear  to  be  more  sensitive  to  trust.  An  Edge  structure  in  a  mistrust 
condition  does  not  appear  able  to  exploit  the  very  benefits  of  Edge.  The  free-form 
communication  and  synergistic  cooperation  that  is  typical  and  sought  after  appears  to  be 
undermined  or  disrupted  in  the  absence  of  organizational  trust  in  the  form  of  perceived 
integrity,  benevolence  and  competence.  In  a  high-trust  condition,  Edge  outperforms  a 
hierarchy  in  speed  and  accuracy.  The  experiment  confirms  that  in  groups  with  high  trust, 
the  Edge  organization  will  solve  problems  with  better  accuracy  and  speed  than  a  mistrust 
condition.  In  general,  problem-solving  effectiveness  is  optimized  in  an  Edge,  Trust 
environment. 

Hierarchy  is  a  more  typical  organization  structure  in  a  staff  headquarters  or 
operational  command.  Tiers  of  power  and  influence  are  often  necessary  boundaries  to 
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allow  managers  to  focus  on  one,  linear  operational  task  at  a  time.  In  general,  Hierarchy  is 
not  as  sensitive  to  trust  and  mistrust.  An  organization’s  problem-solving  effectiveness  in 
a  Hierarchy  would  be  much  the  same  whether  the  organization  has  developed  trust  or  not. 
This  is  actually  a  great  safety  mechanism  in  the  Hierarchy  as  expressed  in  the  operational 
military.  Most  missions  do  not  require  or  allow  for  a  blossoming  of  organizational  trust 
from  the  Commanding  Officer  down  to  the  newest  enlistee  out  of  Basic  Training.  In  a 
Hierarchy,  effectiveness  may  not  explicitly  require  organizational  trust,  in  the  fonn  of 
perceived  integrity,  benevolence  and  competence,  but  trust  is  still  a  goal  worthy  of 
development  through  training  and  culture-management  in  the  military.  The  apparent 
benefit  of  a  Hierarchy  is  its  inherent  and  necessary  compliance  to  the  organizational 
structure  and  roles  in  spite  of  varying  individual  perceptions  of  trust.  In  a  mistrust 
environment,  a  Hierarchy  is  best  suited  to  expedite  complex  problem  solving  and  produce 
precise  results. 

B.  IMPLICATIONS 

To  address  the  most  simple  and  direct  interpretation  of  the  research  and  the 
supporting  literature  best,  the  following  acts  as  a  guide  to  organization  type  and  trust 
conditions. 

In  designing  or  restructuring  an  organization  when  the  luxury  of  being  proactive 
exists,  and  outputs  are  not  urgent: 

•  Commit  to  build  trust  throughout  the  organization,  and  then  form  Edge 
teams  to  solve  specific  problems. 

or 

•  Build  Edge  designed,  cross-functional  teams,  and  then  develop  trust  in  the 
team  members  before  assigning  outputs. 

In  designing  or  restructuring  an  organization  (re-active,  crisis,  urgent): 

•  If  trust  is  absent  or  suspect,  or  the  trust  condition  is  unknown,  and  it  is 
necessary  to  expedite  the  process,  select  a  Hierarchy  organization  with 
clear  roles,  responsibilities,  and  explicitly  define  desired  outcomes. 

•  If  high-trust  is  present  and  it  is  essential  to  must  expedite  the  process,  then 
form  Edge  teams  to  solve  specific  problems. 
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C.  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

Conclusions  and  recommendations  from  this  research  are  based  on  a  substantial 
amount  of  evidence  to  support  the  impact  of  organization  type  and  trust  conditions  on 
organizational  effectiveness.  The  experimental  design  creates  an  easy  set  of  experimental 
outputs  for  analysis,  and  the  evidence  supports  most  of  the  hypotheses.  However,  there  is 
adequate  room  for  improvement  in  the  execution  of  the  design,  and  immense  value  in 
further  research.  The  recommendations  for  further  research  include  the  following. 

Conduct  the  same  basic  experimental  design,  but  provide  more  explicit 
instructions  before  the  ELICIT  session  begins,  or  allow  a  short  “orientation”  to  the 
ELICIT  game.  Ensure  that  moderators  explain  the  nature  of  ELICIT  communication 
protocol  clearly  and  well.  In  particular,  one  could  modify  the  experiment  protocol  to 
ensure  that  all  subjects  know  how  to  Identify  all  specific  aspects  of  the  ELICIT. 

Example: 

“What”  complete  and  correct  answer:  The  Southern  Oil  Pipeline  Terminal 

“What”  answer  given  in  Identify:  Pipeline 

“Where”  complete  and  correct  answer:  Psiland 

“Where”  answer  given  in  Identify:  Terminal  (a  “What”  answer  submitted  as  a 
“Where”) 

“Who”  complete  and  correct  answer:  The  Violet  Group 

“Who”  answer  given  in  Identify:  The  Lion 

“When”  complete  and  correct  answer:  March  15th  at  1100  am 

“When”  answer  given  in  Identify:  The  15th  at  eleven 

Such  refinement  in  protocol  will  likely  result  in  better  accuracy  scores  from  future 
subjects. 

Conduct  the  same  basic  experimental  design,  but  exercise  more  strict  and 
consistent  controls  during  the  subject  interaction.  In  particular,  one  could  modify  the 
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experiment  protocol  to  ensure  that  all  subjects  recognize  that  the  mistrust  condition 
means  that  every  factoid  Shared  and  Posted  is  subject  to  suspicion.  Simply,  future 
experimenters  must  create  conditions  of  mistrust  that  alter  behavior  in  all  subjects. 

Conduct  the  same  basic  experimental  design,  but  lengthen  the  ELICIT  sessions. 
This  will  enable  a  more  gradual  adjustment  to  the  experimental  methods  of 
communication  and  problem  solving.  Even  though  the  complete  set  of  factoids  is 
distributed  within  the  first  20  minutes,  there  are  many  players  who  seem  to  struggle  not 
with  the  infonnation  itself,  but  the  mechanics  of  Share,  Post,  and  Pull.  In  particular,  one 
could  modify  the  experiment  protocol  to  ensure  that  the  ELICIT  game  is  played  for  not 
less  than  90  minutes  or  until  all  subjects  have  Identified  with  confidence  in  their  answer, 
and  without  being  prompted  to  do  so  by  the  moderators.  This  will  also  build  more 
significance  in  the  relationship  between  accuracy  and  speed,  as  the  final  Identify  for  each 
individual  will  be  the  best  estimate  in  the  time  that  actually  elapsed  rather  than  measuring 
the  subject’s  best  estimate  with  a  prompted  Identify  constraint. 

Conduct  the  same  basic  experimental  design,  but  only  accept  time/speed  as  a 
measure  of  effectiveness  for  accurate  and  complete  and  answers.  Certainly,  in  a  problem 
solving  exercise  where  people’s  lives  are  at  risk,  there  should  be  an  expressed  reluctance 
by  subjects  to  submit  hasty  and  reckless  solutions  early  in  the  process  rather  than  accurate 
and  complete  answers  later.  Additionally,  in  the  real  world  of  counterterrorism,  only  an 
answer  that  is  accurate  and  complete  would  suffice  to  allow  the  prevention  of  a  terrorist 
plot  from  occurring. 

Modify  the  experimental  design  to  examine  the  different  components  of  trust — 
perceived  integrity,  benevolence  and  competence — independently  as  well  as  in 
combination.  The  current  design  examines  all  three  components  as  a  single  manipulation, 
but  the  comparative  effects  of  each  component  may  differ.  Further,  trust  can  be  viewed  as 
a  dependent  variable  as  well  as  an  independent  one.  An  additional  experimental  design 
could  examine  the  effect  of  different  organizational  designs,  leadership  styles, 
communication  protocols  and  other  factors  on  the  emergence  and  development  of  trust 
over  time  (e.g.,  with  pretest-posttest  measures). 
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In  summary,  this  study  of  trust  and  its  effect  on  organizations  concludes  that  high 
trust  is  the  preferred  condition  when  Edge  is  selected  over  Hierarchy.  It  is  also 
discovered  that  Edge  Organizations  within  a  high  trust  condition  outperform  other 
organization  and  trust  combinations  when  solving  complex  problems  in  rapidly 
established  groups. 

Factors  of  perceived  trustworthiness  in  this  research  include  ability,  benevolence 
and  integrity.  In  a  military  or  civilian  environment,  trust  is  paramount  to  maximizing 
effectiveness.  In  concert  with  numerous  studies  in  trust  and  organizational  design,  it  is 
found  that  these  factors  are  necessary  personality  traits  for  leaders  to  seek  out  and  to 
develop  in  an  effort  to  establish  and  maintain  trust.  Studies  and  the  results  indicate  that 
other  organizational  dynamics  exist  such  as  the  interaction  between  trust  conditions  and 
organizational  design,  leadership  style,  and  motivational  structure.  This  study  of  trust  in 
differing  organizational  groups  adds  to  the  credible  and  abundant  research  on  trust  and 
organizational  design  and  offers  unique  potential  for  further  experimentation. 
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APPENDIX  A. 


Instructions  to  "A"  Community  (Edge  Trust,  not  shown  to  subjects) 


You  have  been  randomly  assigned  to  the  "A"  community  group.  Your  community  is  a 
group  of  intelligence  officers  whose  task  is  to  reveal  the  identity  of  a  threatening  terrorist 
group,  plot,  location,  and  date/time  of  an  impending  attack.  Working  with  the  members 
for  your  community,  you  will  share  information  and  piece  the  data  together  to  provide  a 
solution. 

Members  of  your  community  share  information  freely  with  a  general  orientation  toward 
doing  good  to  others.  We  are  impressed  with  this  orientation  and  are  encouraged  by  the 
positive  interactions  among  your  fellow  cohort  members.  Moreover,  your  intellect, 
varying  skills,  and  past  experience  lead  us  to  believe  that  you  are  well  qualified  to  solve 
the  terrorist  threat  problem.  We  have  every  reason  to  believe  that  when  it  comes  to 
solving  critical  problems  in  group  settings  such  as  this  that  your  actions  will  be 
consistent,  congruent,  and  credible  with  established  protocols  and  guidelines. 

Group  communications  capabilities  are  available  to  enable  you  to  share  information  with 
any  member  of  your  group.  As  a  member  of  “A”  community,  your  knowledge  and 
unique  experiences  are  valuable  assets,  and  you  are  expected  to  give  best  effort  to 
the  success  of  your  team.  You  are  invited  to  establish  communication  with  any  and 
all  members  of  your  group  and  organize  your  group  in  any  way  that  best  suits  your 
problem  solving. 

Software 


The  software  supports  two  ways  of  informing  Operators  about  factoids  you  have 
“discovered.”  You  can  Share  a  factoid  directly  with  another  subject  using  the  share  tab 
or  you  can  Post  a  factoid  to  one  of  four  websites.  There  are  separate  Who,  What,  Where 
and  When  sites.  Subjects  can  post  intelligence  information  [factoids]  they  are  aware  of  to 
a  website  that  others  in  their  group  can  simultaneously  see  and  update.  Though  these 
areas  are  called  websites,  the  infonnation  display  is  provided  by  the  experiment  software 
and  not  by  the  Internet. 

•  Factoids  in  your  inbox  can  be  copied  into  your  MyFactoids  list  by 
selecting  the  factoid  and  clicking  on  the  Add  to  MyFactoids  action. 

•  POST:  To  Post  a  factoid  to  a  website  that  can  be  viewed  by  your 
Operators,  select  the  factoid  from  your  inbox  or  MyFactoids  list  that  you 
wish  to  post,  click  on  the  Post  action,  and  select  the  website  you  wish  to 
post  to. 
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•  SHARE:  To  Share,  select  the  factoid  from  either  your  inbox  or  your 
MyFactoids  list  that  you  wish  to  share,  click  on  the  Share  action,  and 
select  the  anonymized  name  of  the  person  with  whom  you  want  to  share. 
This  sends  the  factoid  to  the  selected  subject’s  inbox  message  list. 

•  The  Add  to  My  Factoids  action  can  also  be  used  to  copy  a  factoid  from  a 
website  to  your  MyFactoids  list.  To  copy  a  factoid  from  a  website  to  your 
MyFactoids  list,  select  the  factoid  you  wish  to  copy  from  a  website  and 
click  on  the  Add  to  MyFactoids  action. 

In  summary,  a  subject  can  Add  an  item  in  their  inbox  to  their  MyFactoids  list  or  Share  or 
Post  it  directly  with  another  member  of  their  group.  Items  in  their  MyFactoids  list  can 
either  be  Shared  or  Posted  to  a  website.  An  item  on  a  website  can  be  Added  to  a  subject's 
MyFactoids  list. 

"A"  members  also  have  shared  information  tools. 

•  To  get  a  summary  list  of  all  the  factoids  in  your  MyFactoids  list,  click  on 
the  MyFactoids  tab  in  the  middle  of  your  screen. 

•  To  find  out  your  role  information  and  how  other  members  of  your  group 
see  you,  click  on  the  “How  I’m  seen”  tab. 

•  To  get  a  list  of  all  the  members  in  your  group,  with  information  about  their 
role  and  country,  click  on  the  “What  I  see”  tab. 

•  To  access  information  from  your  team  website,  click  on  the  website  that 
you  wish  to  view.  To  update  the  website  with  the  latest  information  that 
has  been  posted  to  it,  click  on  the  Refresh  action  a  the  top  of  the  screen, 
while  viewing  the  website. 

During  each  experiment  round,  you  are  free  to  work  on  any  aspect  of  the  task.  The 
experiment  software  provides  communications  tools  that  you  can  use  to  interact  with  the 
other  members  of  your  group  and  communicate  information. 

When  you  think  that  you  have  identified  the  who,  what,  where  and  when  of  the  adversary 
attack,  click  on  the  Identify  tab  at  the  top  of  your  screen  and  enter  free  text  messages  that 
identify  the  who,  what,  where  and  when  of  an  adversary  attack.  Partial  answers  are 
accepted.  You  may  Identify  more  than  once. 

•  The  who  is  a  group  (for  example  the  blue  group). 

•  The  what  is  a  type  of  target  (for  example  an  embassy  or  religious  school  or 
dignitary). 
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•  The  where  is  the  country  in  which  the  attack  will  take  place  (for  example 
Alphaland). 

•  The  when  is  the  month,  day  and  hour  on  which  the  attack  will  occur  (for 
example  December  15,  at  3:00  am.) 

You  can  re-refer  to  these  instructions  during  the  course  of  the  experiment  by  clicking 
again  on  the  URL  in  the  moderator  message. 

When  you  have  finished  reading  this  important  group  background  infonnation,  and  are 
ready  to  begin  the  practice  round,  click  the  Ready  button  in  the  upper  left  corner  of  your 
screen. 

You  may  have  the  next  10  minutes  to  re-read  these  instructions,  take  a  break  or 
share  recommendations  with  your  team  mates. 
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APPENDIX  B. 


"B"  Community  (Hierarchy  Trust,  not  shown  to  subjects) 


You  have  been  randomly  assigned  to  the  "B"  organizational  group.  Your  community  is  a 
group  of  intelligence  officers  whose  task  is  to  reveal  the  identity  of  a  threatening  terrorist 
group,  plot,  location,  and  date/time  of  an  impending  attack.  Working  with  the  members 
for  your  community,  you  will  share  information  and  piece  the  data  together  to  provide  a 
solution. 

Members  of  your  community  share  information  freely  with  a  general  orientation 
toward  doing  good  to  others.  We  are  impressed  with  this  orientation  and  are 
encouraged  by  the  positive  interactions  among  your  fellow  cohort  members. 
Moreover,  your  intellect,  varying  skills,  and  past  experience  lead  us  to  believe  that 
you  are  well  qualified  to  solve  the  terrorist  threat  problem.  We  have  every  reason  to 
believe  that  when  it  comes  to  solving  critical  problems  in  group  settings  such  as  this 
that  your  actions  will  be  consistent,  congruent,  and  credible  with  established 
protocols  and  guidelines. 

In  the  "B"  organization,  there  are  four  teams  of  four  members  each  plus  an  overall  cross¬ 
team  coordinator.  The  four  teams  are  organized  along  the  lines  of  a  traditional 
hierarchical  command  structure,  each  with  a  leader.  This  diagram  shows  the  hierarchical 
relationship  between  a  Senior  Supervisor  (E5),  the  four  Middle  Managers  (A4,  B4,  C4 
and  D4)  and  their  Operators,  1-3: 


A:  WHO  B:  WHAT  C:  WHERE  D:  WHEN 

Team  A  is  focused  on  Who,  team  B  on  What,  team  C  on  Where  and  team  D  on  When. 
The  overall  coordinator  coordinates  information  between  the  team  leaders  across  team 
boundaries.  Note  that  the  above  diagram  is  an  organization  chart  and  not  a 
communications  chart. 
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Software 


The  software  supports  one  way  of  informing  Operators  about  factoids  you  have 
“discovered.”  You  can  Share  a  factoid  directly  with  another  subject  using  the  Share 
tab.  The  following  restrictions  apply: 

Senior  Supervisors  can  “Share  down”  with  Middle  Managers. 

Operators  can  “Share  across”  with  other  Operators. 

Operators  can  “Share  up”  with  Middle  Managers. 

Middle  Managers  can  “Share  up”  with  the  Senior  Supervisor. 

Each  of  the  4  teams  in  your  group  has  its  own  website.  Subjects  can  post  intelligence 
information  (factoids)  they  are  aware  of  to  a  website  that  others  in  their  group  can 
simultaneously  see  and  post  to.  Though  these  areas  are  called  websites,  the  information 
display  is  provided  by  the  experiment  software  and  not  by  the  Internet.  In  the  "B" 
organization  each  member  of  a  team  can  see  and  update  their  team’s  website.  For 
example,  all  members  of  the  “where”  team  can  see  items  posted  to  their  team’s  “where” 
website.  The  cross-team  coordinator  can  see  all  four  of  the  typed  websites 

•  Factoids  in  your  inbox  can  be  copied  into  your  MyFactoids  list  by 
selecting  the  factoid  and  clicking  on  the  Add  to  MyFactoids  action. 

•  To  Post  a  factoid  to  a  website  that  can  be  viewed  by  your  Operators,  select 
the  factoid  from  your  inbox  or  MyFactoids  list  that  you  wish  to  post,  click 
on  the  Post  action,  and  select  the  website  you  wish  to  post  to. 

•  To  Share,  select  the  factoid  from  either  your  inbox  or  your  MyFactoids  list 
that  you  wish  to  share,  click  on  the  Share  action,  and  select  the 
anonymized  name  of  the  person  with  whom  you  want  to  share.  This  sends 
the  factoid  to  the  selected  subject’s  inbox  message  list. 

•  The  Add  to  My  Factoids  action  can  also  be  used  to  copy  a  factoid  from  a 
website  to  your  MyFactoids  list.  To  copy  a  factoid  from  a  website  to  your 
MyFactoids  list,  select  the  factoid  you  wish  to  copy  from  a  website  and 
click  on  the  Add  to  MyFactoids  action. 

In  summary:  a  subject  can  Add  an  item  in  their  inbox  to  their  MyFactoids  list  or  Share  or 
Post  it  directly  with  another  member  of  their  group.  Items  in  their  MyFactoids  list  can 
either  be  Shared  or  Posted  to  a  website.  An  item  on  a  website  can  be  Added  to  a  subject's 
MyFactoids  list. 
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A"  members  also  have  shared  information  tools. 


•  To  get  a  summary  list  of  all  the  factoids  in  your  MyFactoids  list,  click  on 
the  MyFactoids  tab  in  the  middle  of  your  screen. 

•  To  find  out  your  role  information  and  how  other  members  of  your  group 
see  you,  click  on  the  “How  I’m  seen”  tab. 

•  To  get  a  list  of  all  the  members  in  your  group,  with  information  about  their 
role  and  country,  click  on  the  “What  I  see”  tab. 

•  To  access  information  from  your  team  website,  click  on  the  website  that 
you  wish  to  view.  To  update  the  website  with  the  latest  information  that 
has  been  posted  to  it,  click  on  the  Refresh  action  a  the  top  of  the  screen, 
while  viewing  the  website. 

During  each  experiment  round,  you  are  free  to  work  on  any  aspect  of  the  task.  The 
experiment  software  provides  communications  tools  that  you  can  use  to  interact  with  the 
other  members  of  your  group  and  communicate  information. 

When  you  think  that  you  have  identified  the  who,  what,  where  and  when  of  the  adversary 
attack,  click  on  the  Identify  tab  at  the  top  of  your  screen  and  enter  free  text  messages  that 
identify  the  who,  what,  where  and  when  of  an  adversary  attack.  Partial  answers  are 
accepted.  You  may  Identify  more  than  once. 

•  The  who  is  a  group  (for  example  the  blue  group). 

•  The  what  is  a  type  of  target  (for  example  an  embassy  or  religious  school  or 
dignitary). 

•  The  where  is  the  country  in  which  the  attack  will  take  place  (for  example 
Alphaland). 

•  The  when  is  the  month,  day  and  hour  on  which  the  attack  will  occur  (for 
example  December  15,  at  3:00  am). 

You  can  re-refer  to  these  instructions  during  the  course  of  the  experiment  by  clicking 
again  on  the  URL  in  the  moderator  message. 

When  you  have  finished  reading  this  important  group  background  infonnation,  and  are 
ready  to  begin  the  practice  round,  click  the  Ready  button  in  the  upper  left  corner  of  your 
screen. 

You  may  have  the  next  10  minutes  to  re-read  these  instructions,  take  a  break  or 
share  recommendations  with  your  team  mates. 
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APPENDIX  C. 


"A"  Community  (Edge  Mistrust,  not  shown  to  subjects) 


You  have  been  randomly  assigned  to  the  "A"  community  group.  Your  community  is  a 
group  of  intelligence  officers  whose  task  is  to  reveal  the  identity  of  a  threatening  terrorist 
group,  plot,  location,  and  date/time  of  an  impending  attack.  Working  with  the  members 
for  your  community,  you  will  share  information  and  piece  the  data  together  to  provide  a 
solution. 

Members  of  your  community  normally  work  well  together,  but  frequently  withhold 
information  from  each  other.  We  are  unsure  about  how  you  interact  among  your 
fellow  cohort  members  and  question  whether  negative  interactions  have  affected 
your  relationships.  Moreover,  we  have  yet  to  assess  your  intellect  and  skills,  and 
wonder  whether  past  experience  qualifies  you  to  solve  the  terrorist  threat  problem 
as  a  group.  Previous  sessions  reveal  that  some  individuals  take  pride  in  undermining 
team  cohesion  and  effectiveness  by  generating  and  releasing  false  information  or  by 
non-participation  in  the  exercise.  We  are  discouraged  that  when  it  comes  to  solving 
critical  problems  in  group  settings  such  as  this  that  your  actions  may  not  be 
consistent,  congruent,  and  credible  with  established  protocols  and  guidelines. 
Simply  put,  be  wary  of  moles  and  free-riders. 

Software 


The  software  supports  two  ways  of  informing  Operators  about  factoids  you  have 
“discovered.”  You  can  Share  a  factoid  directly  with  another  subject  using  the  share  tab 
or  you  can  Post  a  factoid  to  one  of  four  websites.  There  are  separate  Who,  What,  Where 
and  When  sites.  Subjects  can  post  intelligence  information  [factoids]  they  are  aware  of  to 
a  website  that  others  in  their  group  can  simultaneously  see  and  update.  Though  these 
areas  are  called  websites,  the  infonnation  display  is  provided  by  the  experiment  software 
and  not  by  the  Internet. 

•  Factoids  in  your  inbox  can  be  copied  into  your  MyFactoids  list  by 
selecting  the  factoid  and  clicking  on  the  Add  to  MyFactoids  action. 

•  POST:  To  Post  a  factoid  to  a  website  that  can  be  viewed  by  your 
Operators,  select  the  factoid  from  your  inbox  or  MyFactoids  list  that  you 
wish  to  post,  click  on  the  Post  action,  and  select  the  website  you  wish  to 
post  to. 

•  SHARE:  To  Share,  select  the  factoid  from  either  your  inbox  or  your 
MyFactoids  list  that  you  wish  to  share,  click  on  the  Share  action,  and 
select  the  anonymized  name  of  the  person  with  whom  you  want  to  share. 
This  sends  the  factoid  to  the  selected  subject’s  inbox  message  list. 
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•  The  Add  to  My  Factoids  action  can  also  be  used  to  copy  a  factoid  from  a 
website  to  your  MyFactoids  list.  To  copy  a  factoid  from  a  website  to  your 
MyFactoids  list,  select  the  factoid  you  wish  to  copy  from  a  website  and 
click  on  the  Add  to  MyFactoids  action. 

In  summary,  a  subject  can  Add  an  item  in  their  inbox  to  their  MyFactoids  list  or  Share  or 
Post  it  directly  with  another  member  of  their  group.  Items  in  their  MyFactoids  list  can 
either  be  Shared  or  Posted  to  a  website.  An  item  on  a  website  can  be  Added  to  a  subject's 
MyFactoids  list. 

"A"  members  also  have  shared  information  tools. 

•  To  get  a  summary  list  of  all  the  factoids  in  your  MyFactoids  list,  click  on 
the  MyFactoids  tab  in  the  middle  of  your  screen. 

•  To  find  out  your  role  information  and  how  other  members  of  your  group 
see  you,  click  on  the  “How  I’m  seen”  tab. 

•  To  get  a  list  of  all  the  members  in  your  group,  with  information  about  their 
role  and  country,  click  on  the  “What  I  see”  tab. 

•  To  access  information  from  your  team  website,  click  on  the  website  that 
you  wish  to  view.  To  update  the  website  with  the  latest  information  that 
has  been  posted  to  it,  click  on  the  Refresh  action  a  the  top  of  the  screen, 
while  viewing  the  website. 

During  each  experiment  round,  you  are  free  to  work  on  any  aspect  of  the  task.  The 
experiment  software  provides  communications  tools  that  you  can  use  to  interact  with  the 
other  members  of  your  group  and  communicate  information. 

When  you  think  that  you  have  identified  the  who,  what,  where  and  when  of  the  adversary 
attack,  click  on  the  Identify  tab  at  the  top  of  your  screen  and  enter  free  text  messages  that 
identify  the  who,  what,  where  and  when  of  an  adversary  attack.  Partial  answers  are 
accepted.  You  may  Identify  more  than  once. 

•  The  who  is  a  group  (for  example  the  blue  group). 

•  The  what  is  a  type  of  target  (for  example  an  embassy  or  religious  school  or 
dignitary). 

•  The  where  is  the  country  in  which  the  attack  will  take  place  (for  example 
Alphaland). 

•  The  when  is  the  month,  day  and  hour  on  which  the  attack  will  occur  (for 
example  December  15,  at  3:00  am.) 

You  can  re-refer  to  these  instructions  during  the  course  of  the  experiment  by  clicking 
again  on  the  URL  in  the  moderator  message. 
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When  you  have  finished  reading  this  important  group  background  infonnation,  and  are 
ready  to  begin  the  practice  round,  click  the  Ready  button  in  the  upper  left  corner  of  your 
screen. 

You  may  have  the  next  10  minutes  to  re-read  these  instructions,  take  a  break  or 
share  recommendations  with  your  team  mates. 
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APPENDIX  D. 


"B"  Community  (Hierarchy  Mistrust,  not  shown  to  subjects) 


You  have  been  randomly  assigned  to  the  "B"  organizational  group.  Your  community  is 
a  group  of  intelligence  officers  whose  task  is  to  reveal  the  identity  of  a  threatening 
terrorist  group,  plot,  location,  and  date/time  of  an  impending  attack.  Working  with  the 
members  for  your  community,  you  will  share  information  and  piece  the  data  together  to 
provide  a  solution. 

Members  of  your  community  normally  work  well  together,  but  frequently  withhold 
information  from  each  other.  We  are  unsure  about  how  you  interact  among  your 
fellow  cohort  members  and  question  whether  negative  interactions  have  affected 
your  relationships.  Moreover,  we  have  yet  to  assess  your  intellect  and  skills,  and 
wonder  whether  past  experience  qualifies  you  to  solve  the  terrorist  threat  problem 
as  a  group.  Previous  sessions  reveal  that  some  individuals  take  pride  in  undermining 
team  cohesion  and  effectiveness  by  generating  and  releasing  false  information  or  by 
non-participation  in  the  exercise.  We  are  discouraged  that  when  it  comes  to  solving 
critical  problems  in  group  settings  such  as  this  that  your  actions  may  not  be 
consistent,  congruent,  and  credible  with  established  protocols  and  guidelines. 
Simply  put,  be  wary  of  moles  and  free-riders. 

In  the  "B"  organization,  there  are  four  teams  of  four  members  each  plus  an  overall  cross¬ 
team  coordinator.  The  four  teams  are  organized  along  the  lines  of  a  traditional 
hierarchical  command  structure,  each  with  a  leader.  This  diagram  shows  the  hierarchical 
relationship  between  a  Senior  Supervisor  (E5),  the  four  Middle  Managers  (A4,  B4,  C4 
and  D4)  and  their  Operators,  1-3: 


A:  WHO  B:  WHAT  C:  WHERE  D:  WHEN 
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Team  A  is  focused  on  Who,  team  B  on  What,  team  C  on  Where  and  team  D  on  When. 
The  overall  coordinator  coordinates  information  between  the  team  leaders  across  team 
boundaries.  Note  that  the  above  diagram  is  an  organization  chart  and  not  a 
communications  chart. 

Software 


The  software  supports  one  way  of  informing  Operators  about  factoids  you  have 
“discovered.”  You  can  Share  a  factoid  directly  with  another  subject  using  the  Share 
tab.  The  following  restrictions  apply: 

Senior  Supervisors  can  “Share  down”  with  Middle  Managers. 

Operators  can  “Share  across”  with  other  Operators. 

Operators  can  “Share  up”  with  Middle  Managers. 

Middle  Managers  can  “Share  up”  with  the  Senior  Supervisor. 

Each  of  the  4  teams  in  your  group  has  its  own  website.  Subjects  can  post  intelligence 
information  (factoids)  they  are  aware  of  to  a  website  that  others  in  their  group  can 
simultaneously  see  and  post  to.  Though  these  areas  are  called  websites,  the  information 
display  is  provided  by  the  experiment  software  and  not  by  the  Internet.  In  the  "B" 
organization  each  member  of  a  team  can  see  and  update  their  team’s  website.  For 
example,  all  members  of  the  “where”  team  can  see  items  posted  to  their  team’s  “where” 
website.  The  cross-team  coordinator  can  see  all  four  of  the  typed  websites 

•  Factoids  in  your  inbox  can  be  copied  into  your  MyFactoids  list  by 
selecting  the  factoid  and  clicking  on  the  Add  to  MyFactoids  action. 

•  To  Post  a  factoid  to  a  website  that  can  be  viewed  by  your  Operators,  select 
the  factoid  from  your  inbox  or  MyFactoids  list  that  you  wish  to  post,  click 
on  the  Post  action,  and  select  the  website  you  wish  to  post  to. 

•  To  Share,  select  the  factoid  from  either  your  inbox  or  your  MyFactoids  list 
that  you  wish  to  share,  click  on  the  Share  action,  and  select  the 
anonymized  name  of  the  person  with  whom  you  want  to  share.  This  sends 
the  factoid  to  the  selected  subject’s  inbox  message  list. 

•  The  Add  to  My  Factoids  action  can  also  be  used  to  copy  a  factoid  from  a 
website  to  your  MyFactoids  list.  To  copy  a  factoid  from  a  website  to  your 
MyFactoids  list,  select  the  factoid  you  wish  to  copy  from  a  website  and 
click  on  the  Add  to  MyFactoids  action. 
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In  summary:  a  subject  can  Add  an  item  in  their  inbox  to  their  MyFactoids  list  or  Share  or 
Post  it  directly  with  another  member  of  their  group.  Items  in  their  MyFactoids  list  can 
either  be  Shared  or  Posted  to  a  website.  An  item  on  a  website  can  be  Added  to  a  subject's 
MyFactoids  list. 

"A"  members  also  have  shared  information  tools. 

•  To  get  a  summary  list  of  all  the  factoids  in  your  MyFactoids  list,  click  on 
the  MyFactoids  tab  in  the  middle  of  your  screen. 

•  To  find  out  your  role  information  and  how  other  members  of  your  group 
see  you,  click  on  the  “How  I’m  seen”  tab. 

•  To  get  a  list  of  all  the  members  in  your  group,  with  information  about  their 
role  and  country,  click  on  the  “What  I  see”  tab. 

•  To  access  information  from  your  team  website,  click  on  the  website  that 
you  wish  to  view.  To  update  the  website  with  the  latest  information  that 
has  been  posted  to  it,  click  on  the  Refresh  action  a  the  top  of  the  screen, 
while  viewing  the  website. 

During  each  experiment  round,  you  are  free  to  work  on  any  aspect  of  the  task.  The 
experiment  software  provides  communications  tools  that  you  can  use  to  interact  with  the 
other  members  of  your  group  and  communicate  information. 

When  you  think  that  you  have  identified  the  who,  what,  where  and  when  of  the  adversary 
attack,  click  on  the  Identify  tab  at  the  top  of  your  screen  and  enter  free  text  messages  that 
identify  the  who,  what,  where  and  when  of  an  adversary  attack.  Partial  answers  are 
accepted.  You  may  Identify  more  than  once. 

•  The  who  is  a  group  (for  example  the  blue  group). 

•  The  what  is  a  type  of  target  (for  example  an  embassy  or  religious  school  or 
dignitary). 

•  The  where  is  the  country  in  which  the  attack  will  take  place  (for  example 
Alphaland). 

•  The  when  is  the  month,  day  and  hour  on  which  the  attack  will  occur  (for 
example  December  15,  at  3:00  am). 

You  can  re-refer  to  these  instructions  during  the  course  of  the  experiment  by  clicking 
again  on  the  URL  in  the  moderator  message. 

When  you  have  finished  reading  this  important  group  background  infonnation,  and  are 
ready  to  begin  the  practice  round,  click  the  Ready  button  in  the  upper  left  corner  of  your 
screen. 
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You  may  have  the  next  10  minutes  to  re-read  these  instructions,  take  a  break  or 
share  recommendations  with  your  team  mates. 

Once  the  moderator  is  ready  to  commence  ELICIT  game  play  and  subjects  are 
appropriately  assigned  logged  in,  subjects  indicate  via  the  ELICIT  client  that  they  are 
ready  to  begin.  Subjects  are  instructed  not  to  reveal  their  pseudonyms  to  one  another 
during  the  game.  Indeed,  they  are  instructed  not  to  talk  or  communicate  with  one  another 
during  the  game  via  any  mechanism  outside  of  the  two  summarized  above  (i.e.,  post-pull, 
share). 

Subjects  are  given  incentives  to  play  the  game  well,  as  participation  and 
performance  are  actual  class-time  requirements,  and  graded.  This  is  intended  to  be  an 
incentive  to  ensure  subjects  cooperate  and  perform  at  their  best.  Each  subject  is  instructed 
to  use  the  Identify  function  as  frequently  as  desired  during  game  play,  but  each  player  in 
the  game  is  expected  to  wait  until  he  or  she  is  relatively  confident  about  the  plot  before 
publishing  their  best  solution  via  the  Identify  command.  The  moderator  prompts  all 
subjects  to  Identify  with  their  best  answer  before  the  game  ends  by  announcing  time 
milestones  at  the  35,  40,  and  44  minute  mark.  The  game  ends  after  45  minutes.  Subjects 
are  not  told  the  results  and  the  correct  answers  to  the  terrorist  plot.  “This  represents  in 
part  the  kind  of  equivocally  inherent  in  intelligence  work:  analysts  are  rarefy  certain 
about  any  suspected  plot,  and  many  are  required  to  work  on  multiple  plots  either 
simultaneously  or  sequentially.  Again,  we  go  to  considerable  lengths  to  enhance  the 
realism  of  the  game — and  hence  external  validity  of  the  results”  (Leweling  &  Nissen, 
2007). 

Controls 

We  have  elected  to  address  the  research  question  with  a  randomized  two-factor 
quasi-experimental  design.  As  noted  above,  the  senior  ranking  person  in  each  group  is 
assigned  as  “Alex,”  and  is  intended  to  play  such  specific  role  through  the  game.  The  role 
of  leadership  is  only  functionally  significant  in  the  hierarchy  organizational  structure 
because  of  the  nature  of  linear  communication  flow.  In  the  edge  organization,  Alex  was 
not  given  any  specific  role  definition  by  the  moderator,  and  therefore  did  not  exert  any 
more  responsibility  than  as  a  spokes-person  accurately  representing  Edge  and  self¬ 
organizing  team  dynamics.  Each  version  of  the  game  is  structurally  equivalent,  and  both 
the  ELICIT  software  and  physical  laboratory  environments  are  consistent  across  the  eight 
experimental  sessions.  To  the  extent  possible,  we  distributed  subjects  into  near-identical 
groups  for  equal  distribution  of  gender,  Service,  military  rank,  and  age,  achieving  the 
greatest  uniformity  between  all  eight  groups. 

Treatment  of  Data 

The  results  of  the  ELICIT  study  will  provide  us  with  one  data  set  per  group.  Each 
individual  subject  will  be  assigned  two  measures  of  effectiveness. 
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1.  Accuracy  rate  of  ELICIT  solution: 

a.  Each  individual  will  receive  a  score  of  1.0  for  each  correct  aspect 
of  the  ELICIT  solution  derived  from  the  Factoids  (Who,  What, 
Where,  When  Month,  When  Day,  and  When  Time). 

b.  The  subject-group’s  accuracy  score  is  the  mean  of  the  17 
participants’  accuracy  scores. 

c.  Calculations: 

((Who  +  What  +  Where)  +  (When  Month  +  When  Day  +  When 
Time)/3)  /4 

((1.0  +  1.0  +  1.0)  +  (0.0  +  1.0  +  0.0)/3)  /4  =  0.83 

2.  Time  in  elapsed  seconds  submission  of  final  ELICIT  solution: 

a.  Each  individual  will  have  elapsed  time  recorded  upon  their  final 
Identify  of  their  ELICIT  solution  derived  from  the  Factoids. 
Baseline  Time  is  established  by  the  maximum  time  elapsed  until 
the  last  individual  submission  of  their  ELICIT  solution  (2872 
seconds) 

b.  The  subject- group’s  accuracy  score  is  the  mean  of  the  17 
participants’  accuracy  scores. 

c.  Calculations: 

(Baseline  -  Individual  Identify  Time)  /  Baseline 
(2872  -  2385)  /  2872  =  0.1695 
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